Fastest rates for stochastic mirror descent methods

Relative smoothness—a notion introduced in Birnbaum et al. (Proceedings of the 12th ACM conference on electronic commerce, ACM, pp 127–136, 2011) and recently rediscovered Bauschke (Math Oper Res 330–348, 2016) Lu (Relatively-smooth convex optimization by first-order methods, applications, arXiv:1610.05708 , 2016)—generalizes standard smoothness typically used analysis gradient type methods. In this work we are taking ideas from well studied field stochastic using them order to obtain faster algorithms for minimizing relatively smooth functions. We propose analyze two new algorithms: Randomized Coordinate Descent (relRCD) Stochastic Gradient (relSGD), both generalizing famous setting. The methods can be fact seen as particular instances mirror descent algorithms, which has been usually analyzed under stronger assumptions: Lipschitzness objective strong convexity reference function. As a consequence, one proposed relRCD corresponds first variant algorithm with linear convergence rate.